Based on the 2000 and 2010 census data, this paper uses a modified gravity model to empirically examine the impact of dialect on inter-provincial migration in China. Our results show that sharing a common dialect has a significant impact on interprovincial migration. Migrant workers tend to move to places with similar dialects and cultural background. We also find that the influence of dialect has decreased over time, and the impact of distance has also dropped due to the improvements in transportation infrastructures. Moreover, we find that the popularity of dialect constrains the inter-provincial labor migration, while the prevalence of mandarin largely eliminates the communication barriers, facilitating the reallocation of labor. Our results suggest that promoting mandarin would encourage population migration and improve the allocation of human capital across regions.